Surgical Oncology 38 (2021) 101637
Available online 27 July 2021
0960-7404/© 2021 Published by Elsevier Ltd.
Performance of image guided navigation in laparoscopic liver surgery – A
systematic review
C. Schneider a,*, M. Allam a,b, D. Stoyanov c,d, D.J. Hawkes d,e, K. Gurusamy a, B.R. Davidson a
a Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK
b General surgery Department, Tanta University, Egypt
c Department of Computer Science, University College London, London, UK
d Centre for Medical Image Computing (CMIC), University College London, London, UK
e Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK
A R T I C L E I N F O
Keywords:
Laparoscopic liver surgery
Laparoscopic liver resection
Robotic liver surgery
Image guided surgery
Computer assisted surgery
Computer assisted navigation
Augmented reality
Machine vision
A B S T R A C T
Background: Compared to open surgery, minimally invasive liver resection has improved short term outcomes. It
is however technically more challenging. Navigated image guidance systems (IGS) are being developed to
overcome these challenges. The aim of this systematic review is to provide an overview of their current capa-
bilities and limitations.
Methods: Medline, Embase and Cochrane databases were searched using free text terms and corresponding
controlled vocabulary. Titles and abstracts of retrieved articles were screened for inclusion criteria. Due to the
heterogeneity of the retrieved data it was not possible to conduct a meta-analysis. Therefore results are presented
in tabulated and narrative format.
Results: Out of 2015 articles, 17 pre-clinical and 33 clinical papers met inclusion criteria. Data from 24 articles
that reported on accuracy indicates that in recent years navigation accuracy has been in the range of 8–15 mm.
Due to discrepancies in evaluation methods it is difficult to compare accuracy metrics between different systems.
Surgeon feedback suggests that current state of the art IGS may be useful as a supplementary navigation tool,
especially in small liver lesions that are difficult to locate. They are however not able to reliably localise all
relevant anatomical structures. Only one article investigated IGS impact on clinical outcomes.
Conclusions: Further improvements in navigation accuracy are needed to enable reliable visualisation of tumour
margins with the precision required for oncological resections. To enhance comparability between different IGS
it is crucial to find a consensus on the assessment of navigation accuracy as a minimum reporting standard.
1. Introduction
Laparoscopic liver resection (LLR) has benefits over open resection in
terms of improved patient recovery, better cosmesis, shorter length of
hospital stay and reduced morbidity [1–5]. Unfortunately complex LLR
such
as
major
hepatectomies
and
segmental
resections
in
superior-posterior segments are technically challenging and have
therefore seen a slow uptake by the surgical community [1,3,6].
A number of factors make LLR technically more challenging than
open resection. The inability to palpate the liver parenchyma makes it
difficult to detect small liver lesions which has caused concerns about
oncological clearance. Because of the liver’s complex three-dimensional
(3D) structure that is derived from its vascular anatomy, it can be
challenging to find and maintain the correct anatomical orientation
within two-dimensional (2D) laparoscopic view which does not provide
depth perception. Poor orientation may lead to incomplete oncological
resection and inadvertent vascular or biliary injury [3,7–10].
Laparoscopic ultrasound (LUS) may be used prior to parenchymal
transection to identify liver lesions and delineate the hepatic vasculature
[11–15]. Once transection has started, however, use of LUS is
demanding because it only provides 2D images which are difficult to
interpret in conjunction with the orientation of the laparoscopic camera.
An additional limitation of LUS is that its diagnostic accuracy is
decreased in the presence of liver cirrhosis, small- or vanishing liver
lesions [8,16–19].
Robotic assisted liver resection has been introduced to overcome the
innate limitations of laparoscopic instruments. Surgical dexterity is
improved by utilisation of endo-wristed instruments with 7◦ of freedom
whereas routine use of stereoscopic laparoscopy enhances depth
* Corresponding author.
E-mail address: crispin.schneider.13@ucl.ac.uk (C. Schneider).
Contents lists available at ScienceDirect
Surgical Oncology
journal homepage: www.elsevier.com/locate/suronc
https://doi.org/10.1016/j.suronc.2021.101637
Received 12 April 2021; Received in revised form 4 July 2021; Accepted 24 July 2021
Surgical Oncology 38 (2021) 101637
2
perception [20]. Similar to LLR however, it is not possible to palpate the
liver and intraoperative interpretation of the 3D anatomical situation is
taxing.
To address these issues image guidance navigation systems (IGS) that
enable intraoperative visualisation of the liver anatomy are being
developed. IGS aim to display anatomical data, spatially correlated to
the operative site, often in the form of 3D models that are created from
cross-sectional imaging. Use of IGS in LLR is particularly appealing
because the display of the highly variable vascular and tumour anatomy
may aid in identifying tumour margins as well as blood vessels and bile
ducts [21,22]. Although IGS are currently widely used in neurosurgery,
orthopaedic surgery and otolaryngology, its evolution in LLR has been
slow [23]. The main obstacles preventing meaningful implementation of
this technology are the mobility of abdominal organs, lack of fixed bony
landmarks for orientation and organ motion secondary to diaphragmatic
and cardiac movement [8,23,24]. Further issues are the paucity of liver
surface features and significant soft tissue deformation due to the
increased intra-abdominal pressure from the pneumoperitoneum and
surgical manipulation [24].
Because of the complexity of the technical challenges a number of
IGS technologies have been developed. These can be broadly categorised
according to the underlying imaging modality into video, ultrasound,
computer tomography (CT) and magnetic resonance imaging (MRI)
-based systems. The aim of this systematic review is to provide a
comprehensive overview of the potential benefits and limitations of IGS
in minimally invasive liver surgery.
2. Methods
A systematic literature search that included the free text and corre-
sponding controlled vocabulary terms for “liver” and “laparoscopy”
combined with those for computer vision terms (e.g. machine vision,
augmented reality), or “image guided surgery” was performed using the
Medline, Embase and Cochrane databases. A detailed description of the
search strategy is stated in Appendix 1. To complement the initial
search, each Medline search term indexed under “Diagnostic Techniques
and Procedures” was screened for relevant image guidance modalities
and included as a separate search term if appropriate.
Full text articles, conference -proceedings and -abstracts describing
in-vivo pre-clinical studies or clinical research on image guidance sys-
tems in minimally invasive liver -resection or -ablation were retrieved.
No backward time restriction was applied to the search and articles
published up to the December 31, 2020 were included.
Exclusion criteria were image guidance for radiotherapy purposes,
ex-vivo research, non-registered image guidance (e.g. preoperative
planning) or non-primary research. No articles were excluded based on
language. Articles reporting on imaging in open liver resection or
laparoscopic cholecystectomy were also excluded. To ensure mid-term
clinical relevance, this review focuses exclusively on in vivo studies.
Systems that do not provide navigation (i.e. lack spatial correlation) are
not reviewed. Screening of the titles and abstracts of retrieved references
was independently carried out by two authors (CS & MA). In case of
disagreement a discussion took place and if the disagreement persisted,
the final decision about inclusion was made by the senior author (BD).
Full texts for eligible articles were retrieved and read. A narrative
summary of the findings is given in table and prose form. Where
possible, system performance is quantified with objective data such as
navigation accuracy and setup time. As the methodology used in the
studies varied significantly no quantitative analysis or meta-analysis
could be conducted.
2.1. General aspects of image guidance in laparoscopic surgery
Most IGS are based on three key components or processes which are:
1) 3D modelling - to create a virtual representation of patient anatomy
2) registration and tracking - to align “virtual” and real anatomy and 3)
Visualisation - to make the information interpretable. 3D modelling is
facilitated by processing volumetric data from CT or MRI scans. For LUS,
CT and MRI -IGS, 3D models are not mandatory since these modalities
have the capability to directly visualise liver anatomy during surgery.
Registration is the technically most challenging step and is thought
to have the greatest impact on navigation accuracy (i.e. how precisely
imaging reflects anatomy). To facilitate registration it is necessary to
obtain biometrical features of the patients liver that can be aligned with
corresponding features on the 3D model. These features may consist of
only a few anatomical landmarks [17] or conversely they may incor-
porate a detailed geometrical liver surface representation [8]. In its most
simple form registration can be carried out manually where the surgeon
aligns 3D model and laparoscopic view [25–29]. Some groups advocate
outlining the liver landmarks with a tracked stylus. Subsequent regis-
tration is achieved by computing the minimum distance between in vivo
and virtual landmarks [8,30]. Laser range scanning may offer an alter-
native method for obtaining biometrical liver data [31].
More recently, semi-automatic registration methods have been
popularised. Most commonly a technique called stereoscopic surface
reconstruction (SSR) that requires a 3D laparoscope also known as a
stereoscope is employed. The right and left video channels of the ste-
reoscope triangulate points on the liver surface (Fig. 1) which are
Abbreviations
AR
augmented reality
CBCT
Cone beam computer tomography
CNN
convolutional neural network
CRLM
colorectal liver metastasis
CT
Computer tomography
FPS
frames per second
IGS
Image guidance system
LLR
Laparoscopic liver resection
LUS
Laparoscopic ultrasound
MRI
Magnetic resonance imaging
SLAM
Simultaneous localisation and mapping
SSR
stereoscopic surface reconstruction
TRE
Target registration error
US
Ultrasound
Fig. 1. Graphic illustrating the concept of SSR. On the left is a 3D laparoscopic
camera with a right and left video channel pointing towards the liver surface on
the right. Viewing the same point through two different spatially fixed video
channels allows calculation of the point-to-camera distance. Reprinted with
permission from Springer Nature [32].
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
3
subsequently amalgamated into a point cloud that is essentially a 3D
points representation of the liver surface. Thereafter a process called ICP
matching is used to align 3D model and point cloud to complete regis-
tration [32].
Tracking provides positional information which enables spatial
correlation between laparoscope, patient anatomy and surgical in-
struments. Optical tracking is the most common method which employs
reflective infrared markers that are attached to instruments [8,33,34].
The position of these markers is recorded by an optical tracking camera
that requires a direct line of sight. This limitation can be avoided by
using electromagnetic (EM) tracking which utilises phase changes
within an EM field to determine positional changes. Calibration is the
process that informs the fixed spatial relationship between tracking
markers and camera optics. Novel concepts such as iterative closest
point (ICP)- and simultaneous localisation and mapping (SLAM)-
tracking are further detailed below.
Earlier systems utilised separate screens to show laparoscopic view
and 3D model next to each other. More recently augmented reality (AR)
displays have been increasingly employed. The advantage of AR is that
patient anatomy and 3D model are visualised on the same screen in an
overlay fashion (Fig. 2). AR is thought to render image interpretation
more intuitive and an additional advantage is that surgical instruments
do not require tracking because they are directly observed within the AR
environment.
Navigation accuracy is often expressed as target registration error
(TRE) which measures how accurately image guidance reflects the
anatomical situation. As a simplification it can be regarded as the sum of
registration- and tracking-error, with the former being the main
contributor to the overall error. Because TRE evaluation is not stand-
ardised, care has to be taken when comparing different IGS [8,24,25]. In
general TRE is calculated by measuring the distance between corre-
sponding landmarks on the 3D model and the patients anatomy.
3. Results
The initial search identified 2015 articles (Fig. 3). Following
screening of titles and abstracts, 1953 articles were excluded. After re-
view of full texts a further 12 articles were excluded, because they either
did not involve in vivo studies (n = 4), studied only cholecystectomy (n
= 1), did not include navigation(n = 3) or were only based on open
surgery (n = 4). Eventually 50 eligible articles, 17 based on preclinical
and 33 based on clinical research were eligible for inclusion. Pre-clinical
research was exclusively conducted on pigs. Information on methodol-
ogy, number of test subjects, key findings and limitations were retrieved
Fig. 2. AR visualisation showing the 3D model overlayed onto the operative site. The liver surface is not displayed to allow a clearer view of blood vessels and bile
ducts (hepatic veins—blue; portal veins—purple; arteries—red, bile ducts & gallbladder—green). (original images by Ref. [75] licensed under CC-BY 4.0). . (For
interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
Fig. 3. Flowchart for selection of articles.
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
4
and summarised in text and table format. To provide an introduction to
the topic and standardise terminology, the results section begins with a
brief description of the key principles underlying IGS and a summary of
relevant findings.
3.1. Video IGS
The first article on Video-IGS published in 2006, investigated laser
range scanning based surface reconstruction in a porcine model [31].
Since this publication there have been no new in vivo studies on this
registration approach and in general most groups prefer to utilise
manual registration with a tracked stylus or user manipulated overlay.
Projecting 3D models externally onto a patients skin may aid laparo-
scopic port placement but visualisation can be altered by ports, in-
struments, and the uneven outline of the abdomen [35].
Currently AR is the most popular visualisation method because, as
demonstrated in a porcine IGS study [36], it is thought to facilitate
mental integration between image guidance data and operative site. The
first clinical report on AR visualisation in LLR was published in 2011
[37]. AR is also a natural fit for robotic assisted liver resection since it
utilises the inherent stereoscopic view of the DaVinci™ [17] console.
3.1.1. Surface reconstruction
Surface reconstruction describes the acquisition of biometric liver
surface characteristics or in other words “reading the liver surface”.
These characteristics can be used for semi-automatic registration but
also to provide data streams to drive modelling of liver deformation (see
below). It has been demonstrated in two porcine studies that semi-
automatic registration is advantageous because it is less time
consuming than manual registration and not influenced by user
dependent registration errors(38,39).
Up to date SSR is the most widely researched surface reconstruction
method. The first in vivo evaluation was published in 2015 on a porcine
model. Using a non-deformable 3D liver model the authors achieved a
TRE≈10 mm. It has been postulated that implementation of a deform-
able 3D model could improve the TRE to approximately 3–4 mm [25].
The application of SSR in humans has been more difficult. Some of the
proposed methods to overcome this issue have been the use of deep
learning to automatically segment (i.e. distinguish) the liver from sur-
rounding organs [40] and the application of a scoring method to identify
the optimal laparoscope position for SSR [29].
SSR can also facilitate tracking without the need for dedicated
tracking equipment. One group proposed the use of ICP tracking, a
method that utilises changes in liver surface biometry to infer laparo-
scope position. Studied in pre-clinical experiments this approach
worked in real-time but navigation accuracy was inferior to that of op-
tical tracking [33].
A potential alternative to SSR is SLAM which is a concept in
computational geometry that enables updating of a map (e.g. liver sur-
face) in an unknown environment while simultaneously tracking objects
[32]. Using a standard monocular laparoscope, it has been demonstrated
in a pre-clinical [41] and a clinical study [42] that SLAM has the po-
tential to enable synchronous tracking and liver surface reconstruction.
3.1.2. Tissue deformation
Most IGS employ a rigid 3D model that cannot adjust shape or po-
sition to reflect physical forces (e.g. respiratory motion, surgical
manipulation) exerted onto the liver. Based on results from porcine
experiments, it has been postulated that deformable liver modelling is
crucial in achieving navigation accuracies of <4 mm [25] and hence
many researchers perceive this to be the holy grail of navigated image
guidance.
The majority of publications are based on retrospective patient video
data [24,43–45] whereas only some groups have attempted intra-
operative evaluation in porcine [46,47] and human [48] studies.
Various models based on complex problem-solving principles in maths
and physics have been postulated but a detailed methodological
description goes beyond the scope of this review.
One of the main obstacles to clinical translation is the substantial
computational expense (i.e. processing power demand), which makes it
challenging to simulate deformable modelling in real-time. Generally,
solutions can be categorised into biomechanical models and data driven
models. The most popular biomechanical solution which has been suc-
cessfully employed in patients, is the finite element method which uti-
lises an organ mesh to represent tissue deformation [43,49]. Potentially
less computationally expensive are data driven models which can be
trained by observing laparoscopic video or synthetic simulations. These
models utilise convolutional neural networks (CNN), a form of machine
learning, which can use graphic processing units to drastically increase
computing speed. It has been suggested that this advantage should
enable real-time functionality in a clinical setting [44]. To the best of our
knowledge however neither biomechanical nor data driven -models
have been able to reliably simulate liver deformation in porcine [47] or
clinical
[43,44]
studies.
In
summary,
AR
visualisation
and
semi-automatic registration are gaining popularity and have the po-
tential to make Video-IGS easier to use. Fundamental improvements to
navigation accuracy will probably depend on the development of reli-
able real-time tissue deformation.
3.2. Laparoscopic ultrasound IGS
One of the greatest obstacles in employing LUS is the difficulty of
mentally integrating 2D US and laparoscopic images. Therefore the
main focus of research has been on developing IGS that integrate LUS
information into the intraoperative environment. The majority of LUS-
IGS utilise B-mode US images as the primary source of visualisation
[40,41] and hence integration of a 3D model is not mandatory. The first
report on LUS-IGS was published in 2014 by a group that overlayed LUS
images onto a 3D laparoscopic video feed in a porcine model. The au-
thors stated that their system facilitated intuitive visualisation of
sub-surface structures [36]. Optical tracking as utilised by this group
cannot be combined with flexible LUS probes since changing the angle of
the probe head is not reflected by the position of the optical tracker. To
address this problem an IGS employing EM tracking markers at the tip of
the LUS probe was developed and evaluated in a pre-clinical study [50].
Another group demonstrated in a clinical setting that LUS images may
also be co-registered with CT images (i.e. correlating LUS images with
spatial location on cross-sectional imaging) to aid in their simultaneous
interpretation(51). It has been shown that LUS-IGS may aid laparoscopic
liver ablation by enabling stereoscopic visualisation of probe trajectory
and tumour position. In a series of 13 patients complete ablation was
achieved in 12 cases [52]. Rather than using LUS for visualisation, one
group demonstrated how it can be utilised for registration instead. Blood
vessel centrelines were acquired with EM tracked LUS in a porcine
model and this data enabled reconstruction of blood vessel anatomy
which subsequently facilitated registration to the corresponding blood
vessels on the 3D model. This approach also enabled integration of LUS
images within the 3D model [53] (Fig. 4). In summary, data so far
suggests that LUS-IGS seem to be particularly useful when co-registered
with CT images or a 3D model. EM tracking is becoming increasingly
popular since it is currently the only viable solution for tracking flexible
LUS probes.
3.3. Computer tomography IGS
CT-IGS have the capacity to acquire volumetric anatomical data (e.g.
liver shape) during surgery. This can then be used for direct visualisation
of liver anatomy or for registration. A crucial step for the advent of CT-
IGS has been an increased availability of cone beam CT (CBCT) within
operating theatres. The first publication on this topic in 2008 reported
the use of optically tracked CBCT during porcine laparoscopy. Regis-
tration of non-contrast and contrast enhanced CBCT was facilitated by
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
5
attaching fiducials to either the skin or the liver surface, respectively.
Following AR visualisation, navigation accuracy was app. 11 mm [54].
Two years later an IGS based on either intermittent or continuous low
dose, non-contrast CT was developed and evaluated in a preclinical
experiment. The low radiation dose of 25 mA enabled regular
re-registration to adapt the 3D model to intraoperative liver deformation
which resulted in a TRE of 1.45 mm. One-off rather than repeat regis-
tration was also explored but this resulted in decreased navigation ac-
curacy since adjustment to liver deformation was not feasible. Major
limitations were increased radiation exposure when using continuous
CT and the requirement for a multi-slice CT scanner within the operating
theatre [55]. Up to date, there has only been one report on CT-IGS
application in a patient. In this report, biometric liver data was ob-
tained by intraoperative CBCT to facilitate registration. Since the
tumour was only visible on MRI, a preoperative MRI was used to process
the 3D liver model. Intraoperative fluoroscopy enabled correlation be-
tween 3D model and surgical instruments [56]. In summary, CT-IGS
technology is a precise registration tool but radiation exposure is high
if it is used for intraoperative cross-sectional imaging.
3.4. Magnetic resonance imaging IGS
MRI guided liver ablation and surgery was made possible by the
invention of the open plane MRI scanner which in contrast to conven-
tional MRI scanners does not completely surround the patient and hence
allows access to conduct procedures. In 2009 a group explored the use of
open plane MRI in a porcine model of LLR. They determined that a T2
weighted sequence with fast spin echo provided the best image quality
while offering an acceptable image acquisition time. An electromag-
netically shielded control room contained all non-MRI compatible
equipment. Within the MR field surgeons used non-ferromagnetic
laparoscopic ports in conjunction with a Nd:YAG laser which enabled
tissue dissection and coagulation. The Nd:YAG titanium manufactured
laser handle was marked with Gadolinium to aid its localisation in MR
images [57]. The only other MRI-IGS study evaluated laparoscopic mi-
crowave ablation. Surgical instruments were constructed from weakly
ferro-magnetic materials. The authors described successful ablation in 6
patients [58]. No 3D models were used in either of these works since
MRI-IGS enabled direct correlation between instruments and liver
anatomy (Fig. 5). In summary, MRI-IGS offers outstanding imaging
quality compared to other IGS modalities but has restrictions in terms of
operating room setup and instrument compatibility.
3.5. Data summary tables
For a table summary of included preclinical and clinical articles
please see (Table 1) and (Table 2), respectively.
4. Discussion
This review has highlighted the current state of the art in navigated
image guidance for minimally invasive liver surgery. The majority of
publications are less than 10 years old which indicates that this tech-
nology is evolving rapidly. IGS have been evaluated in clinical scenarios
right from the inception of this technology, a fact that is reflected by the
large proportion of clinical articles in this review. Most studies were of
Fig. 4. LUS images can be integrated into an
AR 3D model to enhance spatial correlation.
A) LUS probe (in black) examining porcine
liver. B) LUS image (monochrome square
image) is integrated into a 3D porcine liver
model. The position and content of the LUS
image changes when the LUS probe is
moved. Therefore there is spatial correlation
of intrahepatic structures (e.g. blood vessel –
arrow) on LUS image and 3D model. The
liver borders are outlined in grey, hepatic
veins are blue and portal veins are purple.
Tumour locations are shown as yellow le-
sions. (original images by Ref. [53] licensed
under CC-BY 4.0). . (For interpretation of the
references to colour in this figure legend, the
reader is referred to the Web version of this
article.)
Fig. 5. A) Open plane MRI configuration restricts the surgeon’s range of movement. Laparoscopic and MRI images can be visualised by non-ferromagnetic screens
placed at the rear opening of the scanner B) Direct intraoperative visualisation of spatial relationship between the surgeon’s fingers (arrows) and the liver (L). Liver
vessels can be seen as dark circles within the parenchym. reprinted with permission from Springer Nature [85].
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
6
Table 1
Pre-clinical studies
Author &
Journal &
Date and Country
Imaging
modality & No.
of subjects
Study design
& Type of
surgery
Methodology
Important findings
Important limitations
Hayashibe et al. [31]
Medical Image Analysis
August 2006, Japan
Video
n = 1
Exploratory
Laparoscopy
- Registration with laser
surface scanning
- Allows reconstruction of
biometrical liver surface data
in real time.
- Prevents collision of robotic
instruments.
- No registration or
visualisation
demonstrated.
- One subject only.
Konishi et al. [59]
IJCARS
June 2007, Japan
LUS
n = 12
Exploratory
Lap. ablation
- Co-registration of 3D LUS
and video.
- Optical tracking for rigid
instruments.
- EM tracking with
magnetic distortion
correction for flexible
instruments.
- Magnetic distortion
correction improved
navigation accuracy from
17.2 mm to 1.96 mm.
- LUS scanning time app. 30s.
- Time to generate images app.
3 min.
- Lacks comparison of
optical and EM
tracking.
Feuerstein et al. [54]
IEEE Transactions on Medical Imaging
March 2008, Germany
CT-AR
n = 2
Exploratory
LLR
- CBCT used to create 3D
model.
- Display at expiration
only to account for
respiratory motion.
- Navigation accuracy ¼
11.05 ± 4.03 mm.
- Visualisation of major liver
vessels aided in laparoscopic
port placement.
- Respiratory motion increased
TRE by app. 10 mm.
- Unable to visualise
peripheral liver
vessels.
- 3D model lacks detail
Chopra et al. [57]
European Radiology September 2009,
Germany
MRI
n = 2
Exploratory
LLR
- Suitability of different
MR sequences evaluated.
- Development of MR-
compatible theatre setup.
- Optimal MRI sequence is T2
fast spin echo.
- Nitinol built laparoscope is
MR compatible.
- Tissue dissection with 1064-
nm Nd:YAG laser is feasible
and MR-compatible.
- No AR visualisation.
Shekhar et al. [55]
Surgical Endoscopy
August 2010, USA
CT-AR
n = 6
Exploratory
Laparoscopy
- Intraoperative multi-slice
CT (not CBCT).
- Registration with
continuous or non-
continuous low dose non-
contrast CT.
- Navigation accuracy ¼
1.45 mm (low dose) vs.
1.47 mm (high dose).
- Low dose CT reduces
radiation exposure eight fold.
- Continuous scanning enabled
registration updates.
- High radiation
exposure with
continuous CT
compared to CBCT.
Kang et al. [36]
Surgical Endoscopy
July 2014, USA
LUS-AR
n = 2
Exploratory
Laparoscopy
- Overlay of LUS images
onto 3D laparoscopic
view.
- Successful registration of
intrahepatic structures
- Dark tissues (e.g. kidney)
provide better contrast for
overlaying LUS images.
- Feasibility only
demonstrated with
rigid LUS probe.
Thompson et al. [25]
SPIE proceedings
March 2015, UK
Video-AR
(SmartLiver) n
= 5
Exploratory
LLR
- Semi-automatic
registration with SSR
- Accuracy comparison
between rigid and
deformable 3D models.
- Navigation accuracy
app.10 mm.
- Successful registration n = 3/
5.
- Extensive liver deformation
caused failure of SSR.
- Comparison rigid and
deformable 3D models
based on simulation
only.
Reichard et al. [33]
Journal of Medical Imaging
October 2015, Germany
Video-AR
n = 1
Exploratory
Laparoscopy
- SSR registration.
- Comparison of optical
and ICP tracking.
- Navigation accuracy ¼ 13
mm.
- Best accuracy with combined
ICP & optical tracking.
- ICP tracking is more accurate
with HD laparoscope.
- Maximum frame rate 4/s.
- ICP tracking not
working in real-time.
- One subject only.
Song et al. [53]
IJCARS
December 2015, UK
LUS-AR
(SmartLiver) n
= 2
Exploratory
Laparoscopy
- Registration to vascular
landmarks with EM
tracked LUS
- Navigation accuracy ¼
3.7–4.5 mm.
- Accuracy better in proximity
to landmarks.
- LUS images integrated into
3D model.
- No comparison of SSR
vs. LUS registration.
Reichard et al. [47]
IJCARS
July 2017, Germany
Video-AR
n = 1
Exploratory
Laparoscopy
- Semi-automatic
registration with SSR.
- Deformable,
biomechanical 3D liver
model.
- Demonstrated real-time
registration and deformation
on porcine spleen.
- In-vivo data on spleen
only.
- No in vivo accuracy
data.
- One subject only.
Ramalhinho et al. [60]
IJCARS
August 2018, UK
LUS-AR
(SmartLiver) n
= 1
Exploratory
Laparoscopy
- LUS registration as in
Ref. [53].
- Computer simulation to
determine optimal LUS
probe positions for
registration.
- Navigation accuracy ¼
10.4–16.3 mm.
- Higher vascular density in
central liver segments
improves registration.
- Re-evaluation of data
from Ref. [53] but
reports new findings.
- One subject only.
Lau et al. [50]
J Laparoendosc Adv Surg
January 2019, USA
LUS-AR
n = 1
Exploratory
LLR
- EM tracked LUS.
- LUS images overlayed
onto laparoscopic view.
- LLR with AR 7 min. vs. 3 min.
without AR.
- No accuracy data.
- One subject only.
(continued on next page)
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
7
an exploratory nature and were not designed to demonstrate clinical
benefits. This is perhaps unsurprising since at this development stage the
research focus has been on innovation rather than clinical validation.
Twenty-four articles in this review published quantitative data on
navigation accuracy. The methodology of navigation accuracy assess-
ment varies between research groups and therefore it is difficult to
compare results directly [61]. Despite these disparities there appears to
be some evidence that studies using retrospective registration [24,43]
and studies with only one subject [29,66] tend to report better naviga-
tion accuracy which may point towards associated bias. Recently, the
proportion of publications stating accuracy data is increasing, which
perhaps reflects the recognition by scientists that quantifiable data is
paramount to advance the field (Fig. 6).
The advent of AR has been an important development. Whereas
earlier systems relied on two separate screens, AR offers more intuitive
visualisation. Utilisation of AR may cause information overload [70]
which can be addressed by allowing surgeons to switch between full AR,
limited AR (e.g. area of interest, limited opacity) and no AR [70,74].
Enhanced rendering has been proposed as another potential solution
[78]. This technology employs a variety of graphics processing methods
such as plane clipping, distance fogging and shape outlining to focus the
surgeons attention on relevant anatomical details (Fig. 7).
Judging by the number of publications, Video-IGS have seen the
most attention by the research community. This popularity can perhaps
be explained by advantages such as user friendliness, low costs, porta-
bility, high image acquisition speed, and compatibility with existing
surgical equipment [6,23,24]. Its main disadvantage is a lack of depth
penetration which means that the position of deep lying structures can
only be inferred from a 3D model whereas LUS-, CT- and MRI-IGS may
offer direct visualisation of deep structures. Attempts at developing
deformable 3D liver models have been promising [24,44,47] but so far
no group was able to demonstrate real-time functionality during sur-
gery. A previous study estimated that under optimal circumstances a
rigid 3D model could yield a TRE of 8–10 mm. One-off deformation to
adapt to relatively constant changes in liver shape (e.g. after liver
mobilisation) may achieve TRE’s of 5–6 mm whereas real-time soft tis-
sue deformation may further improve the TRE to 2–3 mm [25]. Up to
date, deformation research in LLR has not formally addressed the impact
of liver transection. In open liver surgery it was observed that liver
transection causes up to 8.7 mm displacement of intrahepatic blood
vessels [79]. How this phenomenon will be incorporated into deform-
able 3D liver models for LLR remains to be seen. That deformable 3D
models have so far remained elusive, can perhaps explain why some data
points towards better navigation accuracy for CT and LUS -IGS [53,55,
56,59].
SSR which requires expensive 3D laparoscopes is currently the most
Table 1 (continued)
Author &
Journal &
Date and Country
Imaging
modality & No.
of subjects
Study design
& Type of
surgery
Methodology
Important findings
Important limitations
- Comparing liver
resection margins, AR vs.
no ARAR.
- Clear resection margins in
both groups.
Modrzejewski et al. [46]
IJCARS
April 2019, France
Video-AR
n = 1
Exploratory
Laparoscopy
- Semi-automatic
registration with SSR.
- Deformable,
biomechanical 3D liver
model.
- Various dataset of liver
deformation recorded for
public use.
- Navigation accuracy ¼ 20
mm (intrahepatic) vs. 15
mm (liver surface).
- Self-collision restraint of
deformable 3D model
improved navigation
accuracy by 1–2 mm.
- SSR methodology not
described in detail.
- One subject only.
Luo et al. [61]
Computer Methods and Programs in
Biomedicine
September 2019, China
Video-AR
n = 5
Exploratory
LLR
- Semi-automatic
registration with SSR.
- 3D modelling and
registration with
convolutional neural
networks.
- Liver surface fiducials to
aid registration.
- Navigation accuracy ¼ 8.7
± 2.4 mm -Liver surface
reconstruction and
registration in app. 3 min.
- Frame rate 10–12 fps ex-vivo.
- Review of different accuracy
evaluation methods.
- Navigation not in real-
time.
- In vivo frame rate not
stated.
- Requires
intraoperative CT.
Teatini et al. [38]
Scientific Reports
December 2019, Norway
Video-AR
n = 4
Exploratory
Laparoscopy
- Manual registration.
- Creation and comparison
of pre- (multislice CT)
and intra-operative
(CBCT) 3D models.
- Evaluation fiducials vs.
user-defined landmarks.
- Navigation accuracy ¼
19.04 mm (intraoperative
3D model) vs. 38.37 mm
(preoperative 3D model).
- Landmark dependent error
20.3 mm (manual selection)
vs. 14.38 mm (fiducial).
- Accuracy improved with
minimum 4–5 landmarks.
- Fiducial results only
for three subjects.
- Unclear how visible
diathermy marking is
on CT liver.
Teatini et al. [39]
Min Invasive Ther
Jan 2020, Norway
Video-AR
n = 1
Exploratory
Laparoscopy
- Comparison of manual
registrations by different
surgeons.
- Evaluating impact of
sampling error on
accuracy.
- Navigation accuracy 13.37
± 6.25 mm
- Different accuracy results
between surgeons (p =
0.00045).
- Only one subject
- Usage of different
accuracy metrics is
confusing.
Liu et al. [62]
IJCARS
May 2020, USA
LUS
n = 1
Exploratory
Lap. ablation
- EM tracked LUS
- LUS images showing
needle trajectory
overlayed onto
laparoscopic view.
- IGS feasibility demonstrated.
- Artificial tumours
successfully targeted.
- Comparison of AR vs.
LUS guided needle
placement ex vivo
only.
Table 1. Summary of included preclinical articles. Navigation accuracy data is highlighted in bold. Journal name abbreviations: IJCARS - International Journal of
Computer Assisted Radiology and Surgery; J Laparoendosc Adv Surg - Journal of Laparoendoscopic & Advanced Surgical Techniques; Min Invasive Ther - Minimally
Invasive Therapy & Allied Technologies.
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
8
Table 2
Clinical studies.
Author &
Journal &
Date and Country
Imaging modality
& No. of subjects
Study design
& Type of
surgery
Methodology
Important findings
Important limitations
Volont´e et al. [35]
J Hepatobil Pancreat
Sci
April 2011,
Switzerland
Video-AR (OsiriX)
n = not stated
Exploratory
Robotic
- Manual registration to external
landmarks.
- Projection of 3D model on patient
skin.
- External projection aided in
laparoscopic port placement.
- 3D model distorted by
instruments and ports.
- No accuracy data.
Nicolau et al. [37]
Surgical Oncology
September 2011,
France
Video-AR
n = 5
Exploratory
LLR
- Manual registration.
- Estimated portal vein position AR
vs. surgeon assessment vs. LUS
(control).
- Registration more precise with
small field of view.
- Repeat registration if field of view
changes.
- AR superior to surgeon assessment
in 2/5 cases.
- No accuracy data.
- IGS technology not
described.
Kingham et al. [8]
Journal of
gastrointestinal
surgery
July 2013, USA
Video (Explorer™)
n = 32
Exploratory
Laparoscopy
- Manual registration and additional
laser surface scanning registration
in some open cases.
- Comparison open surgery vs.
laparoscopy at 7mmhg & 14
mmHg.
- Navigation accuracy ¼ 4.9 ± 1.3
mm (laparoscopic at 14 mmHg)
vs. 5.4 ± 2.1 mm (open).
- Accuracy comparable at 7 mmHg
vs. 14 mmHg.
- Registration time app. 3min.
- No performance metrics
for laparoscopic group
- No surgeon feedback.
Buchs et al. [17]
J Surg Res
October 2013,
Switzerland
Video (CAS-One
Surgery™) n = 2
Exploratory
Robotic
- Manual registration.
- AR integrated into robotic console.
- IGS useful for localising lesions.
- Potentially faster manual
registration due to robotic tremor
elimination.
- No accuracy data.
Kenngott et al. [56]
Surgical Endoscopy
March 2014, Germany
CT-AR
n = 1
Exploratory
LLR
- CBCT registration using liver
volume reconstruction.
- 3D model constructed from MRI
since tumour not visible on CT.
- Feasible to determine optimal liver
transection plane.
- No in vivo accuracy data.
- No respiratory gating.
- One subject only.
Satou et al. [26]
Hepatology Int.
March 2014, Japan
Video-AR n = 7
Exploratory
LLR
- Manual registration.
- Intraoperative tumour location
correlated with AR.
- No accuracy data.
- Technology not
described.
Hammill et al. [63]
Surgical Innovation
August 2014, USA
Video (Explorer™)
n = 27
Clin. study
Lap. ablation
- Manual registration
- Comparison LUS vs. IGS ablation
probe placement.
- Navigation accuracy ¼ 19.56
mm.
- Comparable accuracy IGS vs. LUS
(13.15 mm).
- Additional error
introduced by optical
tracking of flexible
ablation probe.
Sindram et al. [52]
HPB
January 2015, USA
LUS
n = 13
Exploratory
Lap. ablation
- EM tracked LUS.
- Ablation probe position and needle
trajectory visualised.
- Clinical evaluation.
− 34 lesions ablated in 13 patients.
- Incomplete ablation n = 1.
- Re-ablation in 7 % (same sitting).
- Clin. Outcomes: complications n
= 3; no mortality.
- No accuracy data.
- No data on early
recurrence.
- No control group.
Pessaux et al. [27]
Langenbeck’s Archives
of Surgery
April 2015, France
Video-AR
n = 3
Exploratory
Robotic
- Manual registration.
- One-off deformation to adjust to
pneumoperitoneum.
- External beam projection of 3D
model.
- AR aided in the identification of
tumour and other structures.
- No accuracy data.
- No surgeon feedback.
Haouchine et al. [43]
IEEE Trans Vis Comput
Graph
May 2015, France
Video-AR
n = 1
Exploratory
Laparoscopy
- Semi-automatic registration with
SSR.
- Deformable, biochemical 3D
model.
- Individual deformation modelling
for liver parenchym and blood
vessels.
- Navigation accuracy app. 4 mm.
- Frame rate of 25 fps.
- Increasing number of 3D model
elements improves accuracy.
- Retrospective
registration.
- Functionality depends on
good initial manual
registration.
- One subject only.
Murakami et al. [58]
Surgery Today
September 2015,
Japan
MRI
n = 6
Exploratory
Lap. ablation
- Designed MR-compatible, weakly
ferromagnetic laparoscope.
- Clinical feasibility demonstrated.
- No significant complications.
- Mean procedure time 275 min.
- Long procedure time.
- No control group.
Plantef`eve et al. [24]
Annals of Biomedical
Engineering
January 2016, France
Video-AR
n = 2
Exploratory
Laparoscopy
- Deformable, biomechanical 3D
model.
- Individual deformation modelling
of parenchym, blood vessels and
Glissonian capsule.
- Landmarks used in addition to
surface registration.
- Navigation accuracy < 1.1 mm.
- Only feasible if 30–40 % of liver
surface is visible.
- Use of landmarks creates
deformation boundaries that
improves registration and 3D
modelling.
- Retrospective
registration.
- Further development
from Ref. [43] but
reports new findings.
Huber et al. [64]
Zeitschrift für
Gastroenterologie
January 2016,
Germany
Video (CAS-One
Surgery™) n = 1
Case report
LLR
- Manual registration.
- 3D model based on CT prior to
neoadjuvant chemotherapy
- Vanished liver lesion (i.e. not
visible on LUS or inspection)
localised by IGS.
- No accuracy data.
- One subject only.
Schneider et al. [65]
HPB
April 2016, UK
Video-AR
(SmartLiver) n =
11
Exploratory
Lap. and LLR
- Manual and semi-automatic regis-
tration with SSR.
- Evaluation of usability.
- Structured surgeon feedback.
- Setup time app. 21 min.
- Feedback suggests the setup
process is too complex.
- No accuracy data
- Part retrospective
analysis.
Conrad et al. [66]
Journal of the
American College of
Video-AR (CAS-
One Surgery™) n
= 1
Case report
LLR
- Manual registration.
- Two-stage hepatectomy.
- Navigation accuracy app. 5 mm.
- One subject only.
- Not compared to LUS.
(continued on next page)
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
9
Table 2 (continued)
Author &
Journal &
Date and Country
Imaging modality
& No. of subjects
Study design
& Type of
surgery
Methodology
Important findings
Important limitations
Surgeons
October 2016, USA
- AR used to guide liver transection
during 1st stage.
- IGS useful for orientation but is
unable to identify all relevant
anatomical structures.
- Registration time app. 1 min.
Aoki et al. [51]
The American Surgeon
December 2016, Japan
LUS
n = 1
Exploratory
LLR
- EM tracked LUS.
- Co-registration of LUS and CT scan.
- Intrahepatic structures manually
highlighted on CT.
- Able to visualise spatial
relationship between surgical
instruments and anatomical
structures.
- No accuracy data.
- IGS technology not
described.
- One subject only.
Robu et al. [29]
IJCARS
July 2017, UK
Video-AR
(SmartLiver) n = 1
Exploratory
LLR
- Semi-automatic registration with
SSR.
- Systematic scoring to evaluate
optimal laparoscope positions for
facilitating SSR.
- Navigation accuracy ¼ 4.7 mm
- Method improved TRE from 17.5
mm to 4.7 mm.
- Identified 4 optimal surface
patches for registration
- Further development
from Ref. [65] but
reports new findings.
- One subject only.
Tinguely et al. [67]
Surgical Endoscopy
October 2017,
Switzerland
Video (CAS-One
Surgery™) n = 51
Clin. study
Lap. ablation
- Manual registration.
- IGS guided liver ablation.
- Evaluation of IGS performance and
clinical outcomes.
- Navigation accuracy ¼ 8.1 mm.
- Successful registration in all
patients.
- Calibration time = 1 min;
Registration time = 4 min.
- Early recurrence n = 16.
- No control group.
- Concomitant bowel or
liver resection in some
patients.
Phutane et al. [68]
Surgical Endoscopy
January 2018, France
Video-AR
n = 1
Video pres.
LLR
- Manual registration.
- Empiric evaluation during major
hepatectomy.
- AR aided identification of
transection plane, middle hepatic
vein and tumour.
- AR less useful during transection
due to organ deformation.
- No accuracy data.
- Only one case described
although 8 cases
performed.
- IGS technology not
described.
Heiselman et al. [49]
Journal of Medical
Imaging
April 2018, USA
Video (Explorer™)
n = 25
Exploratory
Laparoscopy
- Manual registration.
- Deformable, biomechanical 3D
model.
- Liver ligaments and posterior liver
used as fixed points around which
liver deformation is modelled.
- Comparison of deformable and
rigid 3D modelling.
- Navigation accuracy ¼ 14.7 mm
(rigid model) vs. 7.9 mm (Rucker
method) vs. 6.4 mm (deformable
3D model).
- Registration time 140–320s.
- Deformation modelling can be
done preoperatively.
- Frame rate not stated.
- Further development
from Ref. [8] but reports
new findings.
Robu et al. [69]
IJCARS
June 2018, UK
Video-AR
(SmartLiver)
n = 1
Exploratory
LLR
- Semi-automatic registration with
SSR.
- Two step ICP matching
- 1st step coarse registration to
landmark.
- 2nd step fine tuning registration by
SSR.
- Feasibility of 2 step registration
demonstrated.
- Method may form basis for fully
automatic registration without
initial manual alignment.
- No accuracy data.
- Further development
from Ref. [65] but
reports new results.
- One subject only.
Thompson et al. [70]
IJCARS
June 2018, UK
Video-AR
(SmartLiver)
n = 9
Exploratory
Laparoscopy
and LLR
- Manual registration.
- Real-time visual feedback on
navigation accuracy.
- Assessing correlation between
surface landmarks, intrahepatic
structures and navigation accuracy.
- Navigation accuracy app. 12
mm.
- Surface landmarks are reliable
predictors of TRE and suitable
substitutes for intrahepatic
structure localisation.
- Mixed real-time and
retrospective
registration.
- Further development
from Ref. [25] but
reports new results.
Mahmoud et al. [41]
IEEE Trans Med
Imaging July 2018,
France
Video-AR
n = 1
Exploratory
Laparoscopy
- Dense SLAM for registration and
tracking.
- IGS works with monocular
laparoscopes.
- Clinical feasibility of SLAM
demonstrated.
- IGS can adapt to minor
deformation (e.g. respiratory
motion).
- Retrospective
registration.
- No in vivo accuracy data.
- One subject only.
Beerman et al. [71]
European journal of
radiology open
December 2018,
Sweden
Video (CAS-One
Surgery™) n = not
stated
Clin. study
Lap. ablation
- Manual registration.
- Retrospective analysis of IGS
ablation.
- High frequency jet ventilation
reduces undesired respiratory liver
motion.
- IGS improved user confidence
compared to LUS guidance.
- No accuracy data.
- Number of laparoscopic
cases not stated.
- No control group.
Le Roy et al. [72]
J. of Visceral Surgery
February 2019, France
Video-AR
n = 1
Video pres.
LLR
- Semi-automatic registration.
- One-off deformation to adjust 3D
model to intraoperative in vivo liver
shape.
- IGS localised liver lesion which was
not visible on LUS due to artefact.
- Standard monocular laparoscope
used.
- No accuracy data.
- IGS technology not
described.
- One subject only.
Yasuda et al. [73]
Asian Journal of
Endoscopic Surgery
April 2019, Japan
Video-AR
n = 4
Clin. study
LLR
- Manual registration.
- CT cholangiography incorporated
into 3D model.
- Landmarks measured with tape and
marked with diathermy.
- IGS performance compared LLR vs.
open surgery.
- Navigation accuracy ¼ 8.8 mm
(LLR) vs. 7.5 mm (open), (p ¼
0.68).
- Repeat registration improved
deformation error.
- Surgically exposed liver vessels
used as landmarks.
- Adding more landmarks did not
improve accuracy.
- Registration time ≤2 min.
- Accuracy not stated for
individual patients.
- Not clear how additional
landmarks were
registered.
Pfeiffer et al. [44]
IJCARS
April 2019, Germany
Video
n = 1
Exploratory
Laparoscopy
- Deformable, data driven 3D model
based on a convolutional neural
network.
- IGS has potential to adapt
deformation to patient specific
factors (e.g. liver consistency).
- No in vivo accuracy data.
- Retrospective
registration.
- One subject only.
(continued on next page)
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
10
popular solution for semi-automatic registration. Semi-automatic
registration could be expanded to cheaper monocular laparoscopes if
registration through shading and motion or SLAM becomes feasible in
the future [33,41,44,80]. CNN have been successfully used to estimate
position and orientation of objects in a 2D image. At 50–94 frames per
second this method is faster and more accurate than biomechanical
approaches [81]. Since no 3D laparoscopes are required, CNN could
potentially facilitate ICP tracking and semi-automatic registration in
conjunction with monocular laparoscopes.
There are two main applications for LUS-IGS. Firstly it can be
employed as a registration tool to identify subsurface liver structures (e.
g. vessels) which are subsequently registered to a 3D model or CT scan
[51,53]. Secondly it can facilitate integration of LUS images into an AR
display [36,51,53]. Advantages of LUS are wide availability, portability,
low costs, high image acquisition speed and an excellent resolution and
depth penetration. Disadvantages are its inherent 2D nature and user
dependent accuracy. Co-registration of LUS and CT images as standalone
visualisation may offer some advantages over routine LUS but in our
opinion this is unlikely to provide the same benefit as AR with a 3D
model.
There were only three eligible articles on CT-IGS. Two articles
demonstrated CBCT based registration [54,56] whereas the third article
purported low dose spiral CT as a feasible alternative to CBCT [55].
CT-IGS offer good navigation accuracy, visualisation of intrahepatic
structures and the ability to generate volumetric rather than just surface
data. Disadvantages are low resolution (CBCT), ionising radiation, high
costs and lack of portability [56,82]. At this stage, CT-IGS have the best
published navigation accuracy [55,56] which may make them useful as
a benchmarking tool.
Only two publications reported on MRI-IGS, one on liver resection
and liver ablation, respectively. Advantages of this modality are excel-
lent imaging quality and the ability to generate volumetric data.
Table 2 (continued)
Author &
Journal &
Date and Country
Imaging modality
& No. of subjects
Study design
& Type of
surgery
Methodology
Important findings
Important limitations
- Model trained by synthetic data
using multiple organ like meshes.
- Data driven modelling runs at 50
fps.
- No deformation modelling of
surgical manipulation.
Prevost et al. [74]
Journal of
gastrointestinal
surgery
September 2019,
Switzerland
Video-AR (CAS-
One AR™)
n = 10
Clinical study
LLR
- Manual registration.
- Further development from
Ref. [17].
- AR overlayed onto 3D video.
- Hepato-caval confluence and porta
hepatis used as preferred
landmarks due to stable position.
- Navigation accuracy ¼ 9.2 mm.
- Selective visualisation of area of
interest.
- Calibration time 43s; Registration
time 8.50 min.
- IGS aids in localising difficult to
identify liver lesions but lacks
precision to fully navigate
resection.
- Not stated how TRE was
calculated in 3D video
space.
Schneider et al. [75]
Surgical Endoscopy
July 2020, UK
Video-AR
(SmartLiver)
n = 18
Clin. study
LLR
- Semi-automatic registration with
SSR
- Comparison of navigation accuracy
manual vs semi-automatic
registration.
- Training of CNN to recognise liver
surface on video.
- Surgeon feedback forms.
- Navigation accuracy ¼ 10.9 mm
(manual) vs. 13.9 mm (semi-
automatic) (p ¼ 0.158)
- Registration successful in n = 16/
18.
- Automatic liver segmentation using
CNN.
- Setup time (10–15 min) needs
improvement.
- Mixed real-time and
retrospective
registration.
- Further development
from Ref. [70] but
reports new results.
Zhang et al. [42]
Surgical Endoscopy
August 2020, China
Video-AR
n = 64 (30 IGS vs.
34 no IGS)
Clin. study
LLR
- SLAM for surface reconstruction
and tracking.
- Semi-automatic registration with
SLAM.
- Simultaneous visualisation of AR
and near infrared imaging with
ICG.
- Clinical outcome comparison IGS
vs. no IGS.
- Reduced length of stay and blood
loss in IGS group.
- IGS visualisation of tumour margin
27/30.
- IGS aided in identifying
intrahepatic structures and liver
transection line.
- Setup time 30s.
- No accuracy data.
- IGS technology not
described.
Aoki et al. [76]
Journal of
Gastrointestinal
Surgery
September 2020,
Japan
LUS
n = 27
Clin. study
LLR
- EM tracked LUS to CT registration.
- Anatomical colour coding of
structures in CT.
- Navigation accuracy ¼ 12 mm.
- Successful image guidance in 26/
27 cases.
- IGS identified 3 lesions not visible
on LUS.
- Registration time <2min; -Setup
time 7min.
- Patient needs to remain
in neutral table position.
- 3D model available but
not registered to patient.
Bertrand et al. [48]
Surgical Endoscopy
December 2020,
France
Video-AR
(Hepataug) n = 17
Clin. study
LLR
- Deformable, biomechanical 3D
model.
- Semi-automatic registration.
- Further development from
Ref. [72].
- No interruption to workflow
- Good correlation between LUS and
IGS
- Two lesions identified that were
not visible on LUS.
- No data on accuracy or
workflow interruption.
- IGS technology not
described.
Aoki et al. [77]
Surgical Oncology
December 2020, Japan
Video-AR
n = 1
Case report
LLR
- Manual registration.
- AR-guided needle puncture of
portal vein branch.
- Positive ICG staining technique of
liver segments.
- Headset visualisation.
- Portal vein branch accurately
targeted.
- Operative time 285 min.
- No accuracy data
- Registered 3D model
available but not utilised
- Very long procedure
time.
Table 2. Summary of included clinical articles. Published navigation accuracy data is highlighted in bold. Journal abbreviations: IJCARS - International Journal of
Computer Assisted Radiology and Surgery; J Hepatobil Pancreat Sci - Journal of Hepato-Biliary-Pancreatic Sciences; J Surg Res - Journal of Surgical Research;
Hepatology Int. - Hepatology International; J. of Visceral Surgery – Journal of Visceral Surgery; IEEE Trans Vis Comput Graph - IEEE Transactions on Visualisation and
Computer Graphics; IEEE Trans Med Imaging - IEEE Transactions on Medical Imaging.
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
11
Disadvantages are incompatibility with standard surgical equipment,
long image acquisition time, very high costs and limited availability.
Surgical freedom of movement is restricted by the size and shape of the
MRI scanner (Fig. 5).
Four articles, all based on Video-IGS, investigated IGS in robotic
assisted surgery [17,24,27,43]. The feasibility of translating IGS meth-
odology from a laparoscopic [27] or open [17] setting to robotic assisted
surgery has been demonstrated. Compared to robotic assisted surgery,
laparoscopic surgery is more widely disseminated and cheaper [83,84].
Therefore it is probable that most IGS innovations will be developed for
LLR initially and subsequentially transferred to a robotic platform if
clinical benefit is sufficiently incentivising.
A number of limitations have to be taken into account. A meta-
analysis of navigation accuracy would have been useful but since a
variety of TRE calculation methods is used by different groups this was
technically not possible. Because this review exclusively focused on in
vivo studies it is possible that recent developments that were only
evaluated ex vivo are not included. In our experience however the
translation process from ex vivo to clinically relevant IGS research is long
and we found that many ex vivo studies have limited surgical relevance.
In conclusion it is the author’s opinion that due to aforementioned
advantages Video and LUS -IGS have the best potential to be developed
into useful tools for LLR. The navigation accuracy of CT-IGS is user in-
dependent and hence it may prove valuable as a benchmark control for
new IGS technology. A generalised summary for practical considerations
of different IGS modalities is shown in Table 3.
Current IGS technology requires further advances to evolve into a
fully dependable navigation tool [42,64]. To allow effective comparison
Fig. 6. Graphic showing published navigation ac-
curacy of Video-IGS which demonstrates that
reporting of navigation accuracy is becoming
increasingly common. Although different evalua-
tion methods are used there appears to be less
discrepancy between the results of different groups
in recent years. Studies where no intraoperative
registration was carried out have been excluded. If
accuracy values between different groups were
compared then only the best value is stated. *Study
with only one subject.
Fig. 7. Different methods of enhanced
rendering are showcased on the same video
sequence showing the right liver with over-
layed hepatic veins (purple), portal veins
(blue), hepatic arteries (red), liver tumours
(green) and gallbladder (yellow). a) Plane
clipping can show what is inside a structure
– arrow pointing out hepatic vein branch
draining the tumour (purple with green hazy
outline) b) Distance fogging enhances
perception of distance by shading objects
differently – arrow pointing at a segmental portal vein branch whose greater transparency indicates an increased distance from the surgeons viewpoint c) Tradi-
tionally anatomical structures are shown completely filled with colour which makes it impossible to see what is behind a structure. Shape outlining enhances edges
that surround structures to improve 3D scene perception and interpretation – arrow indicating border between tumour and gallbladder. (original images by Ref. [70]
licensed under CC-BY 4.0). . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
Table 3
Characteristics of different IGS modalities.
IGS modality
Navigation accuracy
Availability
Transportability
Costs
Main limitation
Video
+
+++
+++
+
Rigid 3D model
LUS
+
+++
+++
+
2D imaging
CT
++
++
+
++
Ionising radiation exposure &
Rigid 3D model
MRI
+++(#)
+
+
+++
Incompatibility with surgical instruments
Table 3. Shown are practical considerations for each IGS modality discussed in this article. # Navigation accuracy not stated but in principle MRI images visualise the
actual intraoperative situation and therefore account for organ deformation and movement.
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
12
of clinical benefits a standardised approach in the evaluation of navi-
gation accuracy would be beneficial [46,70]. An essential step to facil-
itate this is to encourage interdisciplinary collaboration between
imaging scientists and hepatobiliary surgeons and it is hoped that this
review will contribute to this process.
Funder statement
This publication presents independent research commissioned by the
Health Innovation Challenge Fund (HICF-T4-317), a parallel funding
partnership between the Wellcome Trust and the Department of Health.
The views expressed in this publication are those of the author(s) and
not necessarily those of the Wellcome Trust or the Department of Health.
In addition this work was supported by the Wellcome/EPSRC
[203145Z/16/Z].
Declaration of competing interest
Professor Hawkes is a co-founder of IXICO Ltd. Drs. Schneider and
Allam as well as Profs. Davidson, Gurusamy and Stoyanov have no
conflict of interest to declare.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.suronc.2021.101637.
Author statement
Crispin Schneider: Data curation, Writing manuscript; Moustafa
Allam: Data curation, Revision of manuscript; Danail Stoyanov: Com-
puter science expertise, Review & Editing; Kurinchi Gurusamy: Sys-
tematic review expertise, Methodology; David Hawkes: Medical physics
expertise, Validation, Funding; Brian Davidson: Conceptualization, Su-
pervision, Funding.
References
[1] R. Ciria, D. Cherqui, D.A. Geller, J. Briceno, G. Wakabayashi, Comparative short-
term benefits of laparoscopic liver resection: 9000 cases and climbing, Ann. Surg.
263 (4) (2016 Apr) 761–777.
[2] D. Fuks, F. Cauchy, S. Ft´eriche, T. Nomi, L. Schwarz, S. Dokmak, et al., Laparoscopy
decreases pulmonary complications in patients undergoing major liver resection,
Ann. Surg. (2015) 1.
[3] G. Wakabayashi, D. Cherqui, D.A. Geller, J.F. Buell, H. Kaneko, H.S. Han, et al.,
Recommendations for laparoscopic liver resection, Ann. Surg. 261 (4) (2015)
619–629.
[4] A. El-Gendi, M. El-Shafei, S. El-Gendi, A. Shawky, Laparoscopic versus open
hepatic resection for solitary hepatocellular carcinoma less than 5 cm in cirrhotic
patients: a randomized controlled study, J. Laparoendosc. Adv. Surg. Tech. 28 (3)
(2018 Mar) 302–310.
[5] Å.A. Fretland, V.J. Dagenborg, G.M.W. Bjørnelv, A.M. Kazaryan, R. Kristiansen, M.
W. Fagerland, et al., Laparoscopic versus open resection for colorectal liver
metastases, Ann. Surg. 267 (2) (2017) 1.
[6] I. Dagher, N. O’Rourke, D.A. Geller, D. Cherqui, G. Belli, T.C. Gamblin, et al.,
Laparoscopic major hepatectomy: an evolution in standard of care, Ann Surg 250
(5) (2009) 856–860, 2009/10/07.
[7] J. Kirchberg, C. Reißfelder, J. Weitz, M. Koch, Laparoscopic surgery of liver tumors,
Langenbeck’s Arch. Surg. 398 (2013) 931–938.
[8] T.P. Kingham, S. Jayaraman, L.W. Clements, M.A. Scherer, J.D. Stefansic, W.
R. Jarnagin, et al., Evolution of image-guided liver surgery: transition from open to
laparoscopic procedures, J. Gastrointest. Surg. 17 (7) (2013 Jul) 1274–1282.
[9] X. Cai, Z. Li, Y. Zhang, H. Yu, X. Liang, R. Jin, et al., Laparoscopic liver resection
and the learning curve: a 14-year, single-center experience, Surg. Endosc. (2014)
1–8.
[10] F. Cauchy, D. Fuks, T. Nomi, L. Schwarz, L. Barbier, S. Dokmak, et al., Risk factors
and consequences of conversion in laparoscopic major liver resection, Br. J. Surg.
102 (7) (2015 Jun 1) 785–795.
[11] T. Nomi, D. Fuks, Y. Kawaguchi, F. Mal, Y. Nakajima, B. Gayet, Learning curve for
laparoscopic major hepatectomy, Br. J. Surg. 102 (7) (2015 Jun) 796–804.
[12] T. Ishizawa, A.A. Gumbs, N. Kokudo, B. Gayet, Laparoscopic segmentectomy of the
liver: from segment I to VIII, Ann Surg 256 (6) (2012 Dec) 959–964.
[13] M. Abu Hilal, F. Di Fabio, M. Abu Salameh, N.W. Pearce, Oncological efficiency
analysis of laparoscopic liver resection for primary and metastatic cancer: a single-
center UK experience, Arch. Surg. 147 (1) (2012 Jan) 42–48.
[14] K.T. Nguyen, T.C. Gamblin, D.A. Geller, World review of laparoscopic liver
resection-2,804 patients, Ann Surg 250 (5) (2009 Nov) 831–841, 2009/10/06.
[15] H. Topal, J. Tiek, R. Aerts, B. Topal, Outcome of laparoscopic major liver resection
for colorectal metastases, Surg. Endosc. 26 (9) (2012) 2451–2455.
[16] T.P. Kingham, M.A. Scherer, B.W. Neese, L.W. Clements, J.D. Stefansic, W.
R. Jarnagin, Image-guided liver surgery: intraoperative projection of computed
tomography images utilizing tracked ultrasound, HPB (Oxford) 14 (9) (2012 Sep)
594–603.
[17] N.C. Buchs, F. Volonte, F. Pugin, C. Toso, M. Fusaglia, K. Gavaghan, et al.,
Augmented environments for the targeting of hepatic lesions during image-guided
robotic liver surgery, J. Surg. Res. 184 (2) (2013 Oct) 825–831.
[18] Q.R.J.G. Tummers, F.P.R. Verbeek, J.M. Prevoo H a, A.E. Braat, C.I.M. Baeten, J.
V. Frangioni, et al., First experience on laparoscopic near-infrared fluorescence
imaging of hepatic uveal melanoma metastases using indocyanine green, Surg.
Innovat. 22 (1) (2014 Feb) 20–25.
[19] L. Vigano, A. Ferrero, M. Amisano, N. Russolillo, L. Capussotti, Comparison of
laparoscopic and open intraoperative ultrasonography for staging liver tumours, Br
J Surg 100 (4) (2013) 535–542, 2013/01/23.
[20] R. Montalti, G. Berardi, A. Patriti, M. Vivarelli, R.I. Troisi, Outcomes of robotic vs
laparoscopic hepatectomy: a systematic review and meta-analysis, World J.
Gastroenterol. 21 (27) (2015 Jul) 8441–8451.
[21] N.C. Buchs, F. Volonte, F. Pugin, C. Toso, P. Morel, Three-dimensional laparoscopy:
a step toward advanced surgical navigation, Surg Endosc Other Interv Tech 27 (2)
(2013) 692–693.
[22] D.E. Azagury, M.M. Dua, J.C. Barrese, J.M. Henderson, N.C. Buchs, F. Ris, et al.,
Image-guided surgery, Curr. Probl. Surg. 52 (2015) 476–520.
[23] T. Okamoto, S. Onda, K. Yanaga, N. Suzuki, A. Hattori, Clinical application of
navigation surgery using augmented reality in hepatobiliary pancreatic surgery,
Surg. Today 45 (4) (2015) 397–406.
[24] R. Plantef`eve, I. Peterlik, N. Haouchine, S. Cotin, Patient-specific biomechanical
modeling for guidance during minimally-invasive hepatic surgery, Ann. Biomed.
Eng. 44 (1) (2016 Jan) 139–153, 22.
[25] S. Thompson, J. Totz, Y.Y. Song, S. Johnsen, D. Stoyanov, K. Gurusamy, et al.,
Accuracy validation of an image guided laparoscopy system for liver resection,
SPIE Proc (7) (2015) 9415.
[26] S. Satou, T. Mitsui, R. Ninomiya, M. Komagome, N. Akamatsu, F. Ozawa, et al.,
Image overlay navigation of laparoscopic liver resection, Hepatol Int 8 (2014)
1–405.
[27] P. Pessaux, M. Diana, L. Soler, T. Piardi, D. Mutter, J. Marescaux, Towards
cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy,
Langenbeck’s Arch. Surg. 400 (3) (2014 Apr) 381–385.
[28] C. Schneider, S. Thompson, M.J. Clarkson, D.J. Hawkes, B.R. Davidson, A novel
approach to image guidance in laparoscopic liver surgery, Surg. Endosc. 20 (2015).
Suppl 1.
[29] M.R. Robu, P. Edwards, J. Ramalhinho, S. Thompson, B. Davidson, D. Hawkes, et
al., Intelligent viewpoint selection for efficient CT to video registration in
laparoscopic liver surgery, Int J Comput Assist Radiol Surg 12 (7) (2017 Jul)
1079–1088.
[30] M. Fusaglia, M. Peterhans, D. Wallach, G. Beldi, D. Candinas, S. Weber, Validation
of image overlay accuracy in a instrument guidance system for laparoscopic liver
surgery, Hpb (IHPBA 2012) 14 (2012) 520.
[31] M. Hayashibe, N. Suzuki, Y. Nakamura, Laser-scan endoscope system for
intraoperative geometry acquisition and surgical robot safety management, Med.
Image Anal. 10 (4) (2006 Aug) 509–519.
[32] D. Stoyanov, Surgical vision, Ann. Biomed. Eng. 40 (2) (2012 Feb) 332–345.
[33] D. Reichard, S. Bodenstedt, S. Suwelack, B. Mayer, A. Preukschas, M. Wagner, et
al., Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery, J. Med.
Imaging 2 (4) (2015 Oct 10), 45001.
[34] V.M. Banz, M. Baechtold, S. Weber, M. Peterhans, D. Inderbitzin, D. Candinas,
Computer planned, image-guided combined resection and ablation for bilobar
colorectal liver metastases, World J. Gastroenterol. 20 (40) (2014 Oct)
14992–14996, 28.
[35] F. Volont´e, F. Pugin, P. Bucher, M. Sugimoto, O. Ratib, P. Morel, Augmented reality
and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not
only a matter of fashion, J Hepatobiliary Pancreat Sci. 18 (4) (2011 Jul) 506–509.
[36] X. Kang, M. Azizian, E. Wilson, K. Wu, A.D. Martin, T.D. Kane, et al., Stereoscopic
augmented reality for laparoscopic surgery, Surg. Endosc. 28 (7) (2014 Jul)
2227–2235.
[37] S. Nicolau, L. Soler, D. Mutter, J. Marescaux, Augmented reality in laparoscopic
surgical oncology, Surg Oncol 20 (3) (2011) 189–201.
[38] A. Teatini, E. Pelanis, D.L. Aghayan, R.P. Kumar, R. Palomar, Å.A. Fretland, et al.,
The effect of intraoperative imaging on surgical navigation for laparoscopic liver
resection surgery, Sci. Rep. 9 (1) (2019 Dec) 18687, 10.
[39] A. Teatini, J. P´erez de Frutos, B. Eigl, E. Pelanis, D.L. Aghayan, M. Lai, et al.,
Influence of sampling accuracy on augmented reality for laparoscopic image-
guided surgery, Minim Invasive Ther. Allied Technol. (2020) 1–10.
[40] E. Gibson, M.R. Robu, S. Thompson, P.E. Edwards, C. Schneider, K. Gurusamy, et
al., Deep residual networks for automatic segmentation of laparoscopic videos of
the liver, in: R.J. Webster, B. Fei (Eds.), Prog Biomed Opt Imaging - Proc SPIE, vol.
10135, 2017 Mar, p. 101351M, 3.
[41] N. Mahmoud, T. Collins, A. Hostettler, L. Soler, C. Doignon, J.M.M. Montiel, Live
tracking and dense reconstruction for hand-held monocular endoscopy, IEEE
Trans. Med. Imag. 38 (1) (2019 Jul) 79–89, 13.
C. Schneider et al.
Surgical Oncology 38 (2021) 101637
13
[42] P. Zhang, H. Luo, W. Zhu, J. Yang, N. Zeng, Y. Fan, et al., Real - time navigation for
laparoscopic hepatectomy using image fusion of preoperative 3D surgical plan and
intraoperative indocyanine green fluorescence imaging, Surg. Endosc. 34 (8)
(2020) 3449–3459.
[43] N. Haouchine, S. Cotin, I. Peterlik, J. Dequidt, E. Kerrien, M. Berger, et al., Impact
of soft tissue heterogeneity on augmented reality for liver surgery, IEEE Trans.
Visual. Comput. Graph. 21 (5) (2015 May) 584–597, 1.
[44] M. Pfeiffer, C. Riediger, J. Weitz, S. Speidel, Learning soft tissue behavior of organs
for surgical navigation with convolutional neural networks, Int J Comput Assist
Radiol Surg (2019 Apr) 1–9, 16.
[45] J.S. Heiselman, L.W. Clements, J.A. Collins, J.A. Weis, A.L. Simpson, S.
K. Geevarghese, et al., Characterization and correction of intraoperative soft tissue
deformation in image-guided laparoscopic liver surgery, J. Med. Imaging (2) (2018
Apr) 5, 021203.
[46] R. Modrzejewski, T. Collins, B. Seeliger, A. Bartoli, A. Hostettler, J. Marescaux, An
in vivo porcine dataset and evaluation methodology to measure soft-body
laparoscopic liver registration accuracy with an extended algorithm that handles
collisions, Int J Comput Assist Radiol Surg 14 (7) (2019 May) 1237–1245, https://
doi.org/10.1007/s11548-019-02001-4.
[47] D. Reichard, D. H¨antsch, S. Bodenstedt, S. Suwelack, M. Wagner, H. Kenngott, et
al., Projective biomechanical depth matching for soft tissue registration in
laparoscopic surgery, Int J Comput Assist Radiol Surg 12 (7) (2017) 1101–1110.
[48] L.R. Bertrand, M. Abdallah, Y. Espinel, L. Calvet, B. Pereira, E. Ozgur, et al., A case
series study of augmented reality in laparoscopic liver resection with a deformable
preoperative model, Surg. Endosc. 34 (12) (2020) 5642–5648.
[49] J.S. Heiselman, L.W. Clements, J.A. Collins, J.A. Weis, A.L. Simpson, S.
K. Geevarghese, et al., Characterization and correction of intraoperative soft tissue
deformation in image-guided laparoscopic liver surgery, 14, J Med imaging
(Bellingham, Wash) (2) (2018 Apr) 5, 021203.
[50] L.W. Lau, X. Liu, W. Plishker, K. Sharma, R. Shekhar, T.D. Kane, Laparoscopic liver
resection with augmented reality: a preclinical experience, J. Laparoendosc. Adv.
Surg. Tech. 29 (1) (2019 Jan) 88–93.
[51] T. Aoki, M. Murakami, T. Koizumi, A. Fujimori, Y. Enami, T. Kusano, et al.,
Ultrasound with electromagnetic tracking navigation and image fusion system in
laparoscopic liver surgery: an initial clinical experience, Am. Surg. 82 (12) (2016
Dec) e366–368, 1.
[52] D. Sindram, K.A. Simo, R.Z. Swan, S. Razzaque, D.J. Niemeyer, R.M. Seshadri, et
al., Laparoscopic microwave ablation of human liver tumours using a novel three-
dimensional magnetic guidance system, HPB 17 (1) (2015 Jan) 87–93.
[53] Y. Song, J. Totz, S. Thompson, S. Johnsen, D. Barratt, C. Schneider, et al., Locally
rigid, vessel-based registration for laparoscopic liver surgery, Int J Comput Assist
Radiol Surg 10 (12) (2015 Dec) 1951–1961.
[54] M. Feuerstein, T. Mussack, S.M. Heining, N. Navab, Intraoperative laparoscope
augmentation for port placement and resection planning in minimally invasive
liver resection, IEEE Trans. Med. Imag. 27 (3) (2008 Mar) 355–369.
[55] R. Shekhar, O. Dandekar, V. Bhat, M. Philip, P. Lei, C. Godinez, et al., Live
augmented reality: a new visualization method for laparoscopic surgery using
continuous volumetric computed tomography, Surg Endosc Other Interv Tech 24
(8) (2010 Aug) 1976–1985.
[56] H.G. Kenngott, M. Wagner, M. Gondan, F. Nickel, M. Nolden, A. Fetzer, et al., Real-
time image guidance in laparoscopic liver surgery: first clinical experience with a
guidance system based on intraoperative CT imaging, Surg. Endosc. 28 (3) (2014
Mar) 933–940.
[57] S.S. Chopra, J. Rump, S.C. Schmidt, F. Streitparth, C. Seebauer, G. Schumacher, et
al., Imaging sequences for intraoperative MR-guided laparoscopic liver resection in
1.0-T high field open MRI, Eur. Radiol. 19 (9) (2009 Sep) 2191–2196.
[58] K. Murakami, S. Naka, H. Shiomi, H. Akabori, Y. Kurumi, S. Morikawa, et al., Initial
experiences with MR Image-guided laparoscopic microwave coagulation therapy
for hepatic tumors, Surg. Today 45 (9) (2015 Sep) 1173–1178.
[59] K. Konishi, M. Nakamoto, Y. Kakeji, K. Tanoue, H. Kawanaka, S. Yamaguchi, et al.,
A real-time navigation system for laparoscopic surgery based on three-dimensional
ultrasound using magneto-optic hybrid tracking configuration, Int J Comput Assist
Radiol Surg 2 (1) (2007) 1–10.
[60] J. Ramalhinho, M.R. Robu, S. Thompson, K. Gurusamy, B. Davidson, D. Hawkes, et
al., A pre-operative planning framework for global registration of laparoscopic
ultrasound to CT images, Int J Comput Assist Radiol Surg 13 (8) (2018 Aug)
1177–1186, 2.
[61] H. Luo, D. Yin, S. Zhang, D. Xiao, B. He, F. Meng, et al., Augmented reality
navigation for liver resection with a stereoscopic laparoscope, Comput. Methods
Progr. Biomed. (2019 Oct) 105099, 7.
[62] X. Liu, W. Plishker, T.D. Kane, D.A. Geller, L.W. Lau, J. Tashiro, et al., Preclinical
evaluation of ultrasound-augmented needle navigation for laparoscopic liver
ablation, Int J Comput Assist Radiol Surg 15 (5) (2020) 803–810.
[63] C.W. Hammill, L.W. Clements, J.D. Stefansic, R.F. Wolf, P.D. Hansen, D.A. Gerber,
Evaluation of a minimally invasive image-guided surgery system for hepatic
ablation procedures, Surg. Innovat. 21 (4) (2014 Aug) 419–426.
[64] T. Huber, J. Baumgart, M. Peterhans, S. Weber, S. Heinrich, H. Lang, et al.,
[Computer-assisted 3D-navigated laparoscopic resection of a vanished colorectal
liver metastasis after chemotherapy], Zeitschrift für Gastroenterol 54 (1) (2016
Jan) 40–43.
[65] C. Schneider, S. Thompson, Y. Song, J. Totz, A. Desjardins, K. Gurusamy, et al.,
Preliminary results from a clinical study evaluating a novel image guidance system
for laparoscopic liver surgery, 7, HPB 18 (2017 Jun) e99. Conference Publication.
[66] C. Conrad, M. Fusaglia, M. Peterhans, H. Lu, S. Weber, B. Gayet, Augmented reality
navigation surgery facilitates laparoscopic rescue of failed portal vein
embolization, J. Am. Coll. Surg. 223 (4) (2016) e31–e34.
[67] P. Tinguely, M. Fusaglia, J. Freedman, V. Banz, S. Weber, D. Candinas, et al.,
Laparoscopic image-based navigation for microwave ablation of liver tumors — a
multi-center study, Surg. Endosc. 31 (10) (2017) 4315–4324.
[68] P. Phutane, E. Buc, K. Poirot, E. Ozgur, D. Pezet, A. Bartoli, et al., Preliminary trial
of augmented reality performed on a laparoscopic left hepatectomy, Surg Endosc
Other Interv Tech 32 (1) (2018) 514–515.
[69] M.R. Robu, J. Ramalhinho, S. Thompson, K. Gurusamy, B. Davidson, D. Hawkes, et
al., Global rigid registration of CT to video in laparoscopic liver surgery, Int J
Comput Assist Radiol Surg 13 (6) (2018 Jun) 947–956.
[70] S. Thompson, C. Schneider, M. Bosi, K. Gurusamy, S. Ourselin, B. Davidson, et al.,
In vivo estimation of target registration errors during augmented reality
laparoscopic surgery, Int J Comput Assist Radiol Surg 13 (6) (2018 Jun) 865–874.
[71] M. Beermann, J. Lindeberg, J. Engstrand, K. Galm´en, S. Karlgren, D. Stillstr¨om, et
al., 1000 consecutive ablation sessions in the era of computer assisted image
guidance - lessons learned, Eur J Radiol open 6 (2019) 1–8.
[72] B. Le Roy, E. Ozgur, B. Koo, E. Buc, A. Bartoli, Augmented reality guidance in
laparoscopic hepatectomy with deformable semi-automatic computed tomography
alignment (with video), J. Vis. Surg. 156 (3) (2019 Feb) 261–262, https://doi.org/
10.1016/j.jviscsurg.2019.01.009.
[73] J. Yasuda, T. Okamoto, S. Onda, S. Fujioka, K. Yanaga, N. Suzuki, et al.,
Application of image-guided navigation system for laparoscopic hepatobiliary
surgery, Asian J. Endosc. Surg. 13 (1) (2019 Apr) 39–45, https://doi.org/10.1111/
ases.12696.
[74] G.A. Prevost, B. Eigl, I. Paolucci, T. Rudolph, M. Peterhans, S. Weber, et al.,
Efficiency, accuracy and clinical applicability of a new image-guided surgery
system in 3D laparoscopic liver surgery, J. Gastrointest. Surg. 24 (10) (2019 Oct)
2251–2258, https://doi.org/10.1007/s11605-019-04395-7.
[75] C. Schneider, S. Thompson, J. Totz, Y. Song, M. Allam, M.H. Sodergren, et al.,
Comparison of manual and semi-automatic registration in augmented reality
image-guided liver surgery: a clinical feasibility study, Surg. Endosc. 34 (10) (2020
Oct) 4702–4711, 1.
[76] T. Aoki, D.A. Mansour, T. Koizumi, Y. Wada, Y. Enami, A. Fujimori, et al.,
Laparoscopic liver surgery guided by virtual real-time CT-guided volume
navigation, J. Gastrointest. Surg. (2020) 1–5.
[77] T. Aoki, T. Koizumi, M. Sugimoto, M. Murakami, Holography-guided percutaneous
puncture technique for selective near-infrared fluorescence-guided laparoscopic
liver resection using mixed-reality wearable spatial computer, Surg Oncol 35
(October) (2020) 476–477.
[78] C. Schneider, S. Thompson, K. Gurusamy, M. Clarkson, B. Davidson, Use of
enhanced visualisation methods to decrease the effect of organ motion in image
guided laparoscopic liver surgery, Br. J. Surg. 104 (2017 Jul) 5–243. Suppl(ASGBI
Abstracts 2017).
[79] Y. Uchida, K. Taura, M. Nakao, S. Uemoto, A clinical pilot study of Resection
Process Map: a novel virtual hepatectomy software to visualize the resection
process, case series, Int. J. Surg. 71 (2019 Nov) 36–40, 1.
[80] L. Maier-Hein, P. Mountney, A. Bartoli, H. Elhawary, D. Elson, Groch a, et al.,
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic
surgery, Med. Image Anal. 17 (8) (2013) 974–996.
[81] B. Tekin, S.N. Sinha, P. Fua, Real-time seamless single shot 6D object pose
prediction. In: proceedings of the IEEE computer society conference on computer
vision and pattern recognition, IEEE Computer Society (2018) 292–301.
[82] M. Zijlmans, T. Langø, E.F. Hofstad, C.F.P. Van Swol, A. Rethy, Navigated
laparoscopy – liver shift and deformation due to pneumoperitoneum in an animal
model, Minim Invasive Ther. Allied Technol. 21 (3) (2012 May) 241–248.
[83] V. Packiam, D.L. Bartlett, S. Tohme, S. Reddy, J.W. Marsh, D.A. Geller, et al.,
Minimally invasive liver resection: robotic versus laparoscopic left lateral
sectionectomy, J. Gastrointest. Surg. 16 (12) (2012 Dec) 2233–2238.
[84] C.M. Ho, G. Wakabayashi, H. Nitta, N. Ito, Y. Hasegawa, T. Takahara, Systematic
review of robotic liver resection, Surg Endosc Other Interv Tech 27 (3) (2013 Mar)
732–739.
[85] S.S. Chopra, S.C. Schmidt, R. Eisele, U. Teichgr¨aber, I. Van Der Voort, C. Seebauer,
et al., Initial results of MR-guided liver resection in a high-field open MRI, Surg
Endosc Other Interv Tech 24 (10) (2010) 2506–2512.
C. Schneider et al.